---
title: "Supplemental Information"
author: 
date: 12 December 2019
header-includes:
  - \renewcommand{\thefigure}{S\arabic{figure}}
  - \renewcommand{\thetable}{S\arabic{table}}
  - \usepackage{booktabs}
  - \usepackage{longtable}
output:
  pdf_document:
    toc: true
    toc_depth: 2

---

\clearpage

\listoftables

\listoffigures

\clearpage

```{r hidden, eval=FALSE, echo=FALSE, results="hide"}
# setwd("c:/users/thomas/dropbox/brexit identity/studies/conjoint/analysis")
# rmarkdown::render('political-behavior-si.Rmd', output_file = 'politial-behavior-si.pdf')
```

```{r knitr_options, echo=FALSE, results="hide"}
knitr::opts_chunk$set(
  cache = TRUE, 
  echo=FALSE, 
  results="hold", 
  warning=FALSE, 
  #dev="tiff",
  dpi=600,
  fig.width=10,
  fig.height=10
)
options(scipen = 10)
options(width = 80)
options(knitr.table.format = "latex")
```

```{r packages, echo=FALSE, results="hide", message=FALSE}
library("ggplot2")      # plotting
library("rio")          # data import/export
library("car")          # recoding
library("cregg")        # conjoint analysis and plotting
library("ggrepel")      # labels
library("knitr")        # tables
library("xtable")       # tables
```

```{r load_data, echo=FALSE, results="hide"}
d <- rio::import("../data/LSEResults_Brexit_170428_client.sav")
if (file.exists("../data/conjoint-data-stacked.rds")) {
    conj <- rio::import("../data/conjoint-data-stacked.rds")
} else {
    source("analysis-conjoint-cleaning.R", echo = TRUE, encoding = "UTF-8")
}

# add variable labels
attr(conj$immigration, "label") <- "Immigration Controls"
attr(conj$laws, "label") <- "Legal Sovereignty"
attr(conj$rights, "label") <- "Rights of EU Nationals"
attr(conj$budget, "label") <- "Ongoing Budget Payment"
attr(conj$payment, "label") <- "One-off Payment"
attr(conj$trade, "label") <- "Trade Terms"
attr(conj$border, "label") <- "Ireland/NI Border"
attr(conj$timeline, "label") <- "Timeline"

# add vote choice split
conj$Vote <- factor(ifelse(conj$votedleave == 0, "Remain", "Leave"))
```


\clearpage

# Materials and Methods

This section provides general discussion of conjoint methodology and a justification of features of the design used in this study.

## Context and Logic of Conjoint Designs

Measuring public preferences is commonly approached through survey questionnaires, in which individuals express a degree of favor or disfavor toward a particular object such as a policy, product, or candidate. When objects of evaluation have many features, such as price, size, and brand for a commodity good, it is common to ask about those specific aspects as separate questions. Evaluating the relative importance of the different features, however, becomes empirically challenging. An alternative approach is vignette experiments, in which products are randomly varied along a number of dimensions - again, such as price, size, and brand - and respondents rate the product overall. Comparisons between respondents exposed to slightly different products allow researchers to evaluate how much each feature affected overall evaluations. The difference in evaluations is, due to randomization of features, interpretable as the *average causal effect* of the feature on evaluations, which is easily estimated by a t-test or similar difference-in-means estimator (such as ordinary least squares regression).

Importantly, any features that are not made explicit in the presentation of the object introduce a risk of "confounding". For example, if a product can be completely summarized by price, size, and brand but the experiment only presents price and size, participants in the study may make inferences about the brand from the price and size, leading their evaluations to be based not only on the observed product features but also individually heterogeneous and unobserved assumptions about brand. Thus any vignette study that omits features is at risk of drawing on participants' inferences about the objects of evaluation. This is particularly problematic in public opinion polling about Brexit because members of the public may implicitly condition their evaluations of particular features of Brexit (e.g., immigration) on assumptions about what other policies are likely to accompany a specific immigration policy (e.g., trade policy). The most common example of this would be the European Union's insistence that the "four freedoms" of the Single Market, including "freedom of movement of people" and "freedom of movement of goods" are bundled - depending on what assumptions survey respondents make about this bundling, their evaluations of either immigration policy or trade policy are at risk of being confounded. To the extent that the vignettes include these additional potential sources of confounding, a study can cleanly estimate the effect of each.

A conjoint experiment is a generalization of this vignette approach. Specifically, a conjoint is a high-dimensional factorial experiment that allows researchers to uncover the relative influence of different factors in how people make decisions (Auspurg and Hinz 2015; Hainmueller et al. 2014; Jasso 2006; Wallander 2009). Borrowed from marketing research, where it is used to study purchasing decisions, conjoint methodology has recently become prominent in public opinion research for studying complex opinion formation processes such as support for immigration policies (Bansak et al. 2016; Hainmueller and Hopkins 2015), voting for candidates (Hainmueller et al. 2014) and preferences for labor market reform (Gallego and Marx 2017). In a conjoint study, participants are shown a *series* of vignettes that vary according to a determined set of features, with combinations of features randomly varied. Participants then say how much they like or dislike the described scenario. 

Comparisons of evaluations across vignettes similar in all ways save one can be considered average causal effects of each feature. Regression of evaluations on the full set of discretized features (and possibly their interactions) is a common strategy for estimating the effects of each. Randomization of features, again, ensures causal identification of these average causal effects.

Unlike more simplistic vignette studies, however, conjoints are premised upon the fact that observing every combination of features is unlikely. For example, if objects are varied along 8 dimensions, each of which takes 5 levels, there are 5^8 (390,625) different possible vignettes that can be constructed from the combination of object features. In a typical survey sample, it is not possible to observe all of these levels. In response, three diversions are made from the approach typical of between-respondent vignette experiments. First, participants are typically shown pairs of vignettes as a discrete choice alternative, with a forced choice to favor one versus the other (Hainmueller and Hopkins 2015). This effectively doubles the number of evaluations being made during the study because each chosen alternative is both an unselected or selected alternative. 

Second, participants are typically shown multiple pairs of vignettes in order to increase the effective number of observations of choice. For example, showing 1500 participants 5 pairs of alternatives equates to $1500x5x2=15000$ data points. Because the data are drawn from the same individuals, typical variances are inappropriate (being too small due to clustering of responses within individuals). This is, however, relatively easily addressed by increasing the variance of estimated effects through clustered standard errors in a regression estimation procedure.

Finally, because it is unlikely that there is much data in any "cell" of the factorial experiment, researchers do not focus on the average causal effect of each factor level conditional across all other combinations of features. Instead, Hainmueller and Hopkins (2015) suggest focusing on the *average marginal causal effect* (AMCE), which is - as its name implies - the average unconditional marginal effect of each feature level. AMCEs allow researchers to estimate the average effect of each feature level but not the way(s) that level might interact with levels of other features. If a conjoint entails price, size, and brand, an AMCE would estimate the size of the effect of brand, marginalizing across levels of price and product size (which are randomly and typically uniformly distributed), but would not provide an estimate of the effect of brand for particular product sizes or prices. While different from the average causal effect, the AMCE achieves the objective of disentangling the effects of each object feature from the others.

An alternative way of analyzing the conjoint data is to examine levels of support for each feature, marginalizing across values of the other features. This conveys the same information as the AMCEs but maps that data to a scale that can be understood as ``the proportion of respondents preferring a feature level, if available, factoring in all trade-offs''. Thus, if respondents are indifferent between a feature level and its alternatives, about half (0.5) will select an outcome profile that contains that feature. If they feel positive about that feature, such that it makes them more likely to select an outcome profile containing that feature, values will be closer to 1. Similarly, if they feel negative, values will be closer to zero. These values cannot and should not be interpreted as the raw or unconstrained percentage of respondents supporting a given feature level; instead it measures the degree to which individuals trade-off any given level for all of the others.

## Our Design

Our design followed the emerging paradigm for conjoint studies, entailing a number of features of Brexit negotiation outcomes, and the request that survey participants complete multiple discrete choice tasks. At the beginning of the study, participants completed a few brief background questions, with most demograpic data being drawn from YouGov's profile variables, and then completed five sequential conjoint profile ratings. Each conjoint task presented two alternative Brexit negotiation outcome scenarios and asked participants:

> We are interested in your opinions about possible agreements between Britain and the EU regarding Britain's exit from the EU and future relationship. Please consider the following two possible agreements:

They were then shown two outcomes that varied along eight dimensions. The eight dimensions (attributes) were chosen to cover the most salient aspects of the Brexit negotiations. The selection of levels of each feature was fully randomized, as was the order in which those features was presented in each table. This mitigates the risk of "profile ordering effects" wherein certain features are deemed more important because they are presented first or last in the table. 

Because a conjoint is most useful when in characterizes all relevant decision features (any omitted features potentially confounding the effects of the included features), the choice of dimensions and levels of each is incredibly important.

A screenshot of an example set of profiles as presented to participants is shown below:

![Screenshot](../screenshots/conjoint1a.png)

We focus on the eight dimensions that appear most critical to UK-EU negotiations on Britain's withdrawal from the EU. This is exemplified in the British Prime Minister's letter to the European Council President, Donald Tusk, triggering Article 50 to exit the EU on 29 March 2017. In this letter each of the 8 dimensions are prominent (UK Prime Minister 2017a). Prime Minister May's brief statement to Parliament following the triggering of Article 50 also mentioned these negotiation priorities: immigration ("We will control immigration"), trade ("We will pursue a bold and ambitious free trade agreement with the European Union"), the border between the Republic of Ireland and Northern Ireland ("We want to maintain the common travel area with the Republic of Ireland"), citizens' rights ("We seek to guarantee the rights of EU citizens who are already living in Britain, and the rights of British nationals in other member states as early as we can"), the EU's legal authority in Britain ("We will take control of our own laws and bring an end to the jurisdiction of the European Court of Justice in Britain"), ongoing collaboration with the EU in specific areas ("We hope to continue to collaborate with our European partners in the areas of science, education, research and technology"), and a transition phase for Brexit ("...moving into a phased process of implementation in which Britain") (UK Prime Minister 2017b). 

These dimensions are also the ones that feature in the position papers and speeches by EU negotiators. For example, in the EU's agreed "Terms of Reference for the Article 50 Treaty on European Union negotiations" , they explicitly mention the following issues as central to the first stage of the negotiations: citizens' rights, financial settlement and dialogue on Ireland/Northern Ireland (European Commission 2017). Moreover, press coverage of the Brexit negotiations have centred on the policy dimensions included in our conjoint experiment.

In addition to the choice of dimensions, our conjoint design also involved a careful selection of the 'levels' for each of those features. The levels were designed in such a way as to range between the two most extreme negotiation outcomes: a "Soft Brexit", with continued British membership of the EU's Single Market and Customs Union, and a "No Deal" scenario where negotiations break down before an agreement has been reached. For each of the eight dimensions, we have thus chosen levels that realistically reflect a range of potential outcomes of the Brexit negotiations. In between these two extremes are various gradients that also encompass the negotiation position as it has been broadly outlined by the UK government (UK Department for Exiting the European Union 2017). For example, on immigration the levels range from "No control over EU immigration and similar levels of EU immigration to now" – since continued membership of the Single Market would imply Freedom of Movement (see UK Department for Exiting the European Union 2017) – to "Full control over EU immigration and little to no EU immigration", with four intermediate levels that vary the degree of control over immigration and the level of immigration. Another example pertains to the trade agreement with the EU, where continued membership of the Single Market would imply "Few administrative barriers to trade in goods and services and no tariffs on goods", whereas a breakdown in negotiations would imply that British exports to the EU would be subject to WTO-level tariffs and there was no agreement on trade in services ("Many administrative barriers to trade in goods and services and 5% average tariff on goods").  

These levels of each feature of the agreement were fully randomized from the following set of options:

```{r termlabels, echo=FALSE, results="asis"}
frequencies <- cj_freqs(conj, ~ immigration + laws + rights + budget + payment + trade + border + timeline)
bolding <- unlist(lapply(rev(rle(as.integer(frequencies$feature))$lengths), function(n) c(rep.int("plain", n), "bold")))
print(
  xtable(
    stats::setNames(frequencies[c("feature", "level")], c("Feature", "Level")),
    caption=NULL,
    align = c("l", "l", "p{4in}")
  ),
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```

The labels shown here are identical to those presented to participants. We shorten them in our presentation of results for visual simplicity.

```{r, echo=FALSE, results='hide'}
# shorten the labels for tables and graphs
levels(conj$immigration) <- gsub("over EU immigration and", "/", levels(conj$immigration), fixed = TRUE)
levels(conj$immigration) <- gsub(" (to|than) now", "", levels(conj$immigration))

levels(conj$laws) <- gsub("decisions.*", "ECJ", levels(conj$laws))

levels(conj$rights) <- gsub("Must apply for leave to remain under", "Follow", levels(conj$rights), fixed = TRUE)

#levels(conj$budget)

#levels(conj$payment)

levels(conj$trade) <- gsub("administrative barriers to trade in goods and services and", "admin. barriers /", levels(conj$trade), fixed = TRUE)
levels(conj$trade) <- gsub(" on goods", "", levels(conj$trade), fixed = TRUE)

#levels(conj$border)

#levels(conj$timeline)
```

No constraints were placed on combinations of features or on the pairs of alternatives shown, such that within a given profile pairing the two profiles could be very similar or entirely distinct. The number of profiles shown to each respondent (2 x 5 pairings) means that any given respondent was unlikely to see the full set of feature levels. Respondents were also shown a final "fixed" pairing chosen from a set of three alternatives. Because these are not fully randomized, we do not report these results here.

For each pairing of profiles, respondents answered a total of four questions:

 1. Which of these two outcomes do you prefer? (Outcome A; Outcome B)
 2. How acceptable or unacceptable are each of the outcomes to you personally?  (Completely acceptable; Somewhat acceptable; Somewhat unacceptable; Completely unacceptable)
 3. Which outcome(s) do you think would respect the result of the referendum? (Outcome A would respect the result; Outcome B would respect the result; Both would respect the result; Neither would respect the result; Don't know)
 4. How likely do you think it is that the outcome that you have chosen actually happens? (Very likely; Somewhat likely; Not very likely; Not at all likely; Don't know)

We focus on answers to Question 1 and Question 3 in the paper. Results of the analysis of Question 2 closely mirror those of Question 1. We do not report results for Question 4.

The result of this exercise is information evaluating `r nrow(conj)` respondent-profile pairings (2 profiles x 5 profiles-per-respondent x `r nrow(d)` respondents). The full randomization of conjoint profiles means that we have roughly equal numbers of observations for each level of a given feature.



\clearpage

# Main Results: Policy Support

## AMCEs - Full Sample (Figure 2 in Main Text)

The figure below shows the main results for the entire sample of respondents for Q1 as AMCEs:

```{r q1_analysis_amce, fig.cap="Policy Support AMCEs - Full Sample"}
amces_q1 <- cj(conj, Q1 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, level_order = "descending")
suppressMessages(
  plot(amces_q1, xlim = c(-.25,.25), header_fmt = "%s") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) +
    ggplot2::guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

Effects are estimated using ordinary least squares regresion, with standard errors clustered by respondent. In presenting the results, all effects are estimated and plotted with the "no deal" level of each feature used as the baseline. Plots display point estimates and 95% confidence intervals. These effects can therefore be interpreted as differences relative to that, with positive values indicating support for a less "hard" Brexit feature and negative values indicating opposition for a less "hard" Brexit feature.


\clearpage

Full numerical results for Q1 as AMCEs are as follows:

```{r q1_table_amce, results = "asis"}
amces_q1$amce <- ifelse(is.na(amces_q1[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", amces_q1[["estimate"]], amces_q1[["std.error"]]))
print(
  xtable(
    stats::setNames(amces_q1[c("feature", "level", "amce")], c("Feature", "Level", "AMCE")),
    caption="Policy Support AMCEs - Full Sample",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```

\clearpage

## MMs - Full Sample

The figure below shows the main results for the entire sample of respondents for Q1 as marginal means:

```{r q1_analysis_mm, fig.cap="Poliy Support MMs - Full Sample"}
mm_q1 <- cj(conj, Q1 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, level_order = "descending", estimate = "mm")
suppressMessages(
  plot(mm_q1, xlim = c(0.25,0.75), vline = 0.5, header_fmt = "%s") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) +
    ggplot2::guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

Plots display point estimates and 95% confidence intervals. These marginal means can  be interpreted as the proportion of respondents supporting a proposal when it has a given feature net of all other features.


\clearpage

Full numerical results for Q1 in terms of MMs are as follows:

```{r q1_table, results = "asis"}
mm_q1$mm <- ifelse(is.na(mm_q1[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mm_q1[["estimate"]], mm_q1[["std.error"]]))
print(
  xtable(
    stats::setNames(mm_q1[c("feature", "level", "mm")], c("Feature", "Level", "MM")),
    caption="Policy Support MMs - Full Sample",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```


\clearpage

## MMs Split by Leave/Remain Vote (Figure 3 in Main Text)

Here are the results ordered by feature:

```{r q1_levels_unordered, fig.cap="Policy Support MMs Split by Leave/Remain Vote"}
mms_q1_split <- cj(conj, Q1 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Vote, estimate = "mm", level_order = "descending")
suppressMessages(
  plot(mms_q1_split, group = "Vote", vline = 0.5, xlim = c(.25,.75), header_fmt = "%s") +
    ggplot2::scale_colour_manual("Feature", breaks = c("Leave", "Remain"), values = c("black", "darkgray")) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```


\clearpage

Full numerical results for Q1 split by Leave/Remain in terms of MMs are as follows:

```{r q1_table_split_mm, results = "asis"}
mms_q1_split$formatted <- ifelse(is.na(mms_q1_split[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mms_q1_split[["estimate"]], mms_q1_split[["std.error"]]))
mms_q1_split_wide <- mms_q1_split[1:42,]
mms_q1_split_wide$formatted.leave <- mms_q1_split_wide$formatted
mms_q1_split_remain <- mms_q1_split[43:84,]
mms_q1_split_wide$formatted.remain <- mms_q1_split_remain$formatted
print(
  xtable(
    stats::setNames(mms_q1_split_wide[c("feature", "level", "formatted.remain", "formatted.leave")], c("Feature", "Level", "Remain", "Leave")),
    caption="Policy Support MMs Split by Leave/Remain Vote",
    align = c("l", "l", "p{4in}", "r", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```


\clearpage

## Differences in MMs by Leave/Remain Vote (Figure 4 in Main Text)

An advantage of the marginal means approach is that it is possible to convey subgroup differences without imposing a single feature level as a baseline or reference category. This means it is possible to convey subgroup differences in preferences across all feature levels. Such differences for Q1 are shown here:

```{r q1_levels_diffs, fig.cap="Differences in Policy Support MMs by Leave/Remain Vote"}
mms_q1_diffs <- cj(conj, Q1 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Vote, estimate = "mm_diff", level_order = "descending")
suppressMessages(
  plot(mms_q1_diffs, by = "BY", xlim = c(-.25,.25), header_fmt = "%s") +
    ggplot2::xlab("Estimated Remain - Leave Difference") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) + 
    guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

\clearpage

Full numerical results for the Remain minus Leave difference in MMs are as follows:

```{r q1_levels_diffs_table, results = "asis"}
mms_q1_diffs$formatted <- ifelse(is.na(mms_q1_diffs[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mms_q1_diffs[["estimate"]], mms_q1_diffs[["std.error"]]))
print(
  xtable(
    stats::setNames(mms_q1_diffs[c("feature", "level", "formatted")], c("Feature", "Level", "Difference")),
    caption="Differences in Policy Support MMs by Leave/Remain Vote",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```


\clearpage

# Main Results: Policy Support (Continuous Measure)

These results for Q1 (the discrete choice question reported in the paper) are extremely similar to those for Q2 (the continuous measure of acceptability). 

## AMCEs - Full Sample

The figure below shows the main results for the entire sample of respondents for Q2:

```{r q2_analysis, fig.cap="Policy Support (Continuous Measure) AMCEs - Full Sample"}
amces_q2 <- amce(conj, Q2 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, level_order = "descending")
suppressMessages(
  plot(amces_q2, xlim = c(-.25,.25), header_fmt = "%s") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) +
    ggplot2::guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

\clearpage

Full numerical results for Q2 are as follows:

```{r q2_table, results = "asis"}
amces_q2$formatted <- ifelse(is.na(amces_q2[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", amces_q2[["estimate"]], amces_q2[["std.error"]]))
print(
  xtable(
    stats::setNames(amces_q2[c("feature", "level", "formatted")], c("Feature", "Level", "AMCE")),
    caption="Policy Support (Continuous Measure) AMCEs - Full Sample",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```

\clearpage

## MMs - Full Sample

```{r q2_analysis_mm, fig.cap="Policy Support (Continuous Measure) MMs - Full Sample"}
mm_q2 <- cj(conj, Q2 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, level_order = "descending", estimate = "mm")
suppressMessages(
  plot(mm_q2, vline = mean(conj$Q2, na.rm = TRUE), xlim = c(.25,0.75), header_fmt = "%s") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) +
    ggplot2::guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

\clearpage

Full numerical results for Q2 in terms of MMs are as follows:

```{r q2_table_mm, results = "asis"}
mm_q2$formatted <- ifelse(is.na(mm_q2[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mm_q2[["estimate"]], mm_q2[["std.error"]]))
print(
  xtable(
    stats::setNames(mm_q2[c("feature", "level", "formatted")], c("Feature", "Level", "MM")),
    caption="Policy Support (Continuous Measure) MMs - Full Sample",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```

\clearpage

## MMs Split by Leave/Remain Vote

The following pages contain the same alternative analysis approach for (Q2). Importantly, a difference in interpretation is necessary for these analyses. Because respondents were able to independently rate the two scenarios, these results will not center at 0.50. Instead, they are bounded strictly by 0 and 1 but can take on almost any value.

Here are the Q2 results ordered by feature:

```{r q2_levels_unordered, fig.cap="Policy Support (Continuous Measure) MMs Split by Leave/Remain Vote"}
mms_q2_split <- cj(conj, Q2 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Vote, estimate = "mm", level_order = "descending")
suppressMessages(
  plot(mms_q2_split, group = "Vote", vline = mean(conj$Q2, na.rm = TRUE), xlim = c(.25,.75), header_fmt = "%s") +
    ggplot2::scale_colour_manual("Feature", breaks = c("Leave", "Remain"), values = c("black", "darkgray")) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

\clearpage

Full numerical results for Q2 split by Leave/Remain in terms of MMs are as follows:

```{r q2_table_split_mm, results = "asis"}
mms_q2_split$formatted <- ifelse(is.na(mms_q2_split[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mms_q2_split[["estimate"]], mms_q2_split[["std.error"]]))
mms_q2_split_wide <- mms_q2_split[1:42,]
mms_q2_split_wide$formatted.leave <- mms_q2_split_wide$formatted
mms_q2_split_remain <- mms_q2_split[43:84,]
mms_q2_split_wide$formatted.remain <- mms_q2_split_remain$formatted
print(
  xtable(
    stats::setNames(mms_q2_split_wide[c("feature", "level", "formatted.remain", "formatted.leave")], c("Feature", "Level", "Remain", "Leave")),
    caption="Policy Support (Continuous Measure) MMs Split by Leave/Remain Vote",
    align = c("l", "l", "p{4in}", "r", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```

\clearpage

## Differences in MMs by Leave/Remain Vote

Here are the results for Q2:

```{r q2_levels_diffs, fig.cap="Differences in Policy Support (Continuous Measure) MMs by Leave/Remain Vote"}
mms_q2_diffs <- cj(conj, Q2 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Vote, estimate = "mm_diff", level_order = "descending")
suppressMessages(
  plot(mms_q2_diffs, by = "BY", xlim = c(-.25,.25), header_fmt = "%s") +
    ggplot2::xlab("Estimated Remain - Leave Difference") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) + 
    guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

\clearpage

Full numerical results for the Remain minus Leave difference in MMs are as follows:

```{r q2_levels_diffs_table, results = "asis"}
mms_q2_diffs$formatted <- ifelse(is.na(mms_q2_diffs[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mms_q2_diffs[["estimate"]], mms_q2_diffs[["std.error"]]))
print(
  xtable(
    stats::setNames(mms_q2_diffs[c("feature", "level", "formatted")], c("Feature", "Level", "Difference")),
    caption="Differences in Policy Support (Continuous Measure) MMs by Leave/Remain Vote",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```



\clearpage

# Main Results: Perceived Legitimacy

## AMCEs - Full Sample

Here are the overall results for the "respect the referendum" question (Q3) in terms of AMCEs:

```{r q4_analysis, fig.cap="Perceived Legitimacy AMCEs - Full Sample"}
amces_q4 <- amce(conj, Q4 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, level_order = "descending")
suppressMessages(
  plot(amces_q4, xlim = c(-.3,.3), header_fmt = "%s") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) +
    ggplot2::guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

These effects can be interpreted similarly, with positive values indicating a feature *better* respects the referendum result than the "no deal" baseline and negative values indicating a feature *worse* respects the referendum result.


\clearpage

Full numerical results for Q3 as AMCEs are as follows:

```{r q4_table, results = "asis"}
amces_q4$formatted <- ifelse(is.na(amces_q4[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", amces_q4[["estimate"]], amces_q4[["std.error"]]))
print(
  xtable(
    stats::setNames(amces_q4[c("feature", "level", "formatted")], c("Feature", "Level", "AMCE")),
    caption="Perceived Legitimacy AMCEs - Full Sample",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```


\clearpage

## MMs - Full Sample

```{r q4_analysis_mm, fig.cap="Perceived Legitimacy MMs - Full Sample"}
mm_q4 <- cj(conj, Q4 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, level_order = "descending", estimate = "mm")
suppressMessages(
  plot(mm_q4, vline = mean(conj$Q4, na.rm = TRUE), xlim = c(.05,0.55), header_fmt = "%s") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) +
    ggplot2::guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```


\clearpage

Full numerical results for Q3 as MMs are as follows:

```{r q4_table_mm, results = "asis"}
mm_q4$formatted <- ifelse(is.na(mm_q4[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mm_q4[["estimate"]], mm_q4[["std.error"]]))
print(
  xtable(
    stats::setNames(mm_q4[c("feature", "level", "formatted")], c("Feature", "Level", "MM")),
    caption="Perceived Legitimacy MMs - Full Sample",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```


\clearpage

## MMs Split by Leave/Remain Vote (Figure 5 in Main Text)

The following pages contain the same analysis for the ``respect'' question (Q3). Importantly, a difference in interpretation is necessary for these analyses. Because respondents were able to say that either option A, option B, both, or neither choice respected the referendum, these results will not center at 0.50. Instead, they are bounded strictly by 0 and 1 but can take on almost any value.

Here are the Q3 results ordered by feature:

```{r q4_levels_unordered, fig.cap="Perceived Legitimacy MMs Split by Leave/Remain Vote"}
mms_q4_split <- cj(conj, Q4 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Vote, estimate = "mm", level_order = "descending")
suppressMessages(
  plot(mms_q4_split, group = "Vote", vline = mean(conj$Q4, na.rm = TRUE), xlim = c(.05,.55), header_fmt = "%s") + 
    ggplot2::scale_colour_manual("Feature", breaks = c("Leave", "Remain"), values = c("black", "darkgray")) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

\clearpage

Full numerical results for Q2 split by Leave/Remain in terms of MMs are as follows:

```{r q4_table_split_mm, results = "asis"}
mms_q4_split$formatted <- ifelse(is.na(mms_q4_split[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mms_q4_split[["estimate"]], mms_q4_split[["std.error"]]))
mms_q4_split_wide <- mms_q4_split[1:42,]
mms_q4_split_wide$formatted.leave <- mms_q4_split_wide$formatted
mms_q4_split_remain <- mms_q4_split[43:84,]
mms_q4_split_wide$formatted.remain <- mms_q4_split_remain$formatted
print(
  xtable(
    stats::setNames(mms_q4_split_wide[c("feature", "level", "formatted.remain", "formatted.leave")], c("Feature", "Level", "Remain", "Leave")),
    caption="Perceived Legitimacy MMs Split by Leave/Remain Vote",
    align = c("l", "l", "p{4in}", "r", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```

\clearpage

## Differences in MMs by Leave/Remain Vote

Here are the results for Q3:

```{r q4_levels_diffs, fig.cap="Differences in Perceived Legitimacy MMs by Leave/Remain Vote"}
mms_q4_diffs <- cj(conj, Q4 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Vote, estimate = "mm_diff", level_order = "descending")
suppressMessages(
  plot(mms_q4_diffs, by = "BY", xlim = c(-.25,.25), header_fmt = "%s") +
    ggplot2::xlab("Estimated Remain - Leave Difference") +
    ggplot2::scale_colour_manual("Feature", values = rep("black", 8)) + 
    guides(colour=FALSE) +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
)
```

\clearpage

Full numerical results for the Remain minus Leave difference in MMs are as follows:

```{r q4_levels_diffs_table, results = "asis"}
mms_q4_diffs$formatted <- ifelse(is.na(mms_q4_diffs[["std.error"]]), "--", sprintf("%0.2f (%0.2f)", mms_q4_diffs[["estimate"]], mms_q4_diffs[["std.error"]]))
print(
  xtable(
    stats::setNames(mms_q4_diffs[c("feature", "level", "formatted")], c("Feature", "Level", "Difference")),
    caption="Differences in Perceived Legitimacy MMs by Leave/Remain Vote",
    align = c("l", "l", "p{4in}", "r")
  ),
  size = "\\footnotesize",
  floating = FALSE,
  booktabs = TRUE,
  include.rownames = FALSE,
  tabular.environment = "longtable",
  comment = FALSE,
  hline.after = c(-1, 0, cumsum(rle(as.integer(frequencies$feature))$lengths))
)
```


\clearpage 

# Comparison of Support and Perceived Legitimacy of Feature Levels

This figure compares the between-group differences in marginal means for support (forced choice measure) and perceived legitimacy between Leave and Remain voters.

```{r diffs_scatter, fig.width=8, fig.height=8, fig.cap="Comparison of Support and Perceived Legitimacy of Feature Levels"}
merged_diffs <- merge(mms_q1_diffs, mms_q4_diffs, by = c("feature", "level"), suffixes = c(".Q1", ".Q4"))
ggplot(merged_diffs, 
       aes(
         x = estimate.Q1, 
         y = estimate.Q4, 
         colour = feature, 
         label = stringr::str_wrap(level, 25))) + 
  geom_vline(xintercept = 0, colour = "gray70") + 
  geom_hline(yintercept = 0, colour = "gray70") +
  geom_abline(slope = 1, intercept = 0, colour = "gray80") +
  #geom_smooth(method = "lm", colour = "gray80", fill = "gray80", alpha = 0.5) + 
  geom_point(size = 1.5) + 
  # 2-se
  geom_segment(aes(x = estimate.Q1, xend = estimate.Q1, 
                   y = lower.Q4, yend = upper.Q4),
               size = 0.5, alpha = 0.5) + 
  geom_segment(aes(x = lower.Q1, xend = upper.Q1, 
                   y = estimate.Q4, yend = estimate.Q4), 
               size = 0.5, alpha = 0.5) + 
  ggrepel::geom_text_repel(colour = "black", size = 2, force = 10) +
  guides(colour = guide_legend(ncol = 2L)) +
  xlim(c(-0.15, 0.20)) + ylim(c(-0.10, 0.20)) +
  xlab("Difference in Support (Remain - Leave)") + 
  ylab("Difference in Perceived Legitimacy (Remain - Leave)") + 
  theme_bw() + 
  theme(
    #legend.position = c(0.2,0.85),
    legend.position = "none",
    legend.title=element_blank(),
    panel.grid.major = element_blank(),
    panel.grid.minor = element_blank()
  ) +
  coord_fixed() +
  facet_wrap(~ feature, ncol = 3)
```


\clearpage

# Results by 2015 General Election Vote

This section reports conditional results, wherein the conjoint is analyzed conditional on respondent vote choice in 2015 General Election.

## Policy Support

```{r split_by_party1, fig.cap="Policy Support MMs Split by 2015 General Election Vote"}
conj$Party <- rio::characterize(conj$pastvote_2015)
conj$Party[!conj$Party %in% c("Conservative", "Labour", "Liberal Democrat", 
                          "United Kingdom Independence Party (UKIP)")] <- "Other"
conj$Party[conj$Party == "United Kingdom Independence Party (UKIP)"] <- "UKIP"
conj$Party <- factor(conj$Party)
plot(cj(conj, Q1 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Party, estimate = "mm", level_order = "descending"),
     group = "BY", vline = 0.5, header_fmt = "%s") +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
```

## Policy Support (Continuous Measure)

```{r split_by_party2, fig.cap="Policy Support (Continuous Measure) MMs Split by 2015 General Election Vote"}
plot(cj(conj, Q2 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Party, estimate = "mm", level_order = "descending"),
     group = "BY", vline = mean(conj$Q2, na.rm = TRUE), header_fmt = "%s") +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
```

## Perceived Legitimacy

```{r split_by_party3, fig.cap="Perceived Legitimacy MMs Split by 2015 General Election Vote"}
plot(cj(conj, Q4 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Party, estimate = "mm", level_order = "descending"),
     group = "BY", vline = mean(conj$Q4, na.rm = TRUE), header_fmt = "%s") +
    theme(
      axis.text.y = element_text(face = bolding),
      axis.title = element_text(size = 14, face = "bold"),
      legend.text = element_text(size = 14)
    )
conj$Party <- NULL
```

\clearpage

# Supplemental Results

This section contains a number of supplemental analyses of the conjoint design including basic distributions and robustness checks of the conjoint results split by survey characteristics: namely, the order of the profiles shown to respondents and the left/right position of the pairs of profiles on the page. The analysis in each case is based upon using subgroup marginal means based on the characteristic. 


## Sample Construction and Characteristics

```{r demographics}
demog <- d[c("age", "gender", "region_GOR", "voted2015", 
             "pastvote_2015", "pastvote_EURef", "education_level")]
demog$gender <- factor(demog$gender, attributes(demog$gender)$labels, names(attributes(demog$gender)$labels))
demog$region <- factor(demog$region_GOR, attributes(demog$region_GOR)$labels, names(attributes(demog$region_GOR)$labels))
demog$educ <- factor(demog$education_level, attributes(demog$education_level)$labels, names(attributes(demog$education_level)$labels))
demog$pastvote_2015 <- factor(demog$pastvote_2015, attributes(demog$pastvote_2015)$labels, names(attributes(demog$pastvote_2015)$labels))
```

The data for this project were collected by YouGov UK as part of their online Omnibus panel. Respondents are recruited to be representative of the UK adult (18+) population with respect to age, sex, education, political orientation, and geographic region using quota sampling from the Omnibus online panel. All respondents completed the study online over a 24 hour period 26-27 April 2017. A total of `r nrow(d)` respondents completed the study.

The age distribution of the same ranged from `r min(demog$age)` to `r max(demog$age)`, with a median age of `r median(demog$age)`. The full distribution is as follows:

```{r agedist, fig.height=4, fig.width=4, fig.cap="Respondent Age Distribution", fig.align="center"}
ggplot(demog, aes(x = age)) + 
  geom_histogram(binwidth = 3) + 
  ggtitle("Age distribution") + 
  xlab("Age, in years") +
  ylab("Count") +
  theme_bw()
```

\clearpage

The sample was nearly evenly split between men (`r sprintf("%0.1f", 100*prop.table(table(demog$gender)))[1]`%) and women (`r sprintf("%0.1f", 100*prop.table(table(demog$gender)))[2]`%). Respondents were regionally distributed as follows:

\begin{center}

```{r regdist}
knitr::kable(
cbind(data.frame(table(demog$region)), 
      Perc = data.frame(sprintf("%0.1f", 100*prop.table(table(demog$region))))[[1]])
)
```

\end{center}

Politically, `r sprintf("%0.1f", 100*sum(demog$voted2015==1)/nrow(demog))`% reported voting in the 2015 UK General Election, with a distribution of vote choices that closely mirrored the official results:

\begin{center}

```{r votedist}
knitr::kable(
cbind(data.frame(table(demog$pastvote_2015)), 
      Perc = data.frame(sprintf("%0.1f", 100*prop.table(table(demog$pastvote_2015))))[[1]],
      Official = c("36.8", "27.6", "7.9", "4.7", "0.6", "12.6", "3.8", "0.0", "6.0", NA, NA, NA)
))
```

\end{center}

With respect to the EU referendum, `r sum(demog$pastvote_EURef %in% 1:2)` (`r sprintf("%0.1f", 100*sum(demog$pastvote_EURef %in% 1:2)/nrow(demog))`%) of respondents reported voting (substantially higher than the official turnout of 72% of registered voters or 65% of the voting age population). Of those reporting voting, `r sprintf("%0.1f", 100*sum(demog$pastvote_EURef == 1)/sum(demog$pastvote_EURef %in% 1:2))`% voted Remain (against the official result of 48.1%). While this differs slightly from the official result, our analysis focuses separately leave and remain voters rather than sample-level characteristics.



\clearpage

## Feature and Outcome Distributions

This section provides some descriptive statistics from the study and reproduces the main results. The source code for this document fully reproduces the reported figures. We briefly summarize the distributions of responses to the main outcome questions. The data here and in all subsequent analyses is a "stacked" dataset, wherein each alternative from each profile pair is a row, such that the dataset has `r nrow(conj)` rows. As stated above, the levels of features were fully randomized and the following plot of numbers of appearances of each level indicates that within each feature this randomization was successful:

```{r randomizations, fig.width=12, fig.height=8, fig.cap="Feature Level Distribution"}
plot(
  cj_freqs(conj, ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID),
  header_fmt = "%s") +
theme(
    axis.text.y = element_text(face = bolding),
    axis.title = element_text(size = 14, face = "bold"),
    legend.text = element_text(size = 14)
  )
```

We now turn to distributions of outcome measures. Given the forced choice nature of Q1, this means that there are `r nrow(conj)/2L` 0's and `R nrow(conj)/2L` 1's for Q1. Respondents were about 1\% less likely to select the righthand option (`r sprintf("diff=%0.2f (se=%0.2f)", summary(lm(Q1 ~ AB, data = conj))$coef[2,1], summary(lm(Q1 ~ AB, data = conj))$coef[2,2])`) but that did not affect their judgments of whether either option respected the referendum (`r sprintf("diff=%0.2f (se=%0.2f)", summary(lm(Q4 ~ AB, data = conj))$coef[2,1], summary(lm(Q4 ~ AB, data = conj))$coef[2,2])`). Interestingly, respondents indicated, on average, that only `r sprintf("%0.2f", 100*prop.table(table(conj$Q4))[2])`\% of profiles respected the referendum.


\clearpage

## Results by Survey Feature

This section provides some simple robustness checks on the overall results, checking whether results are affected by trivial features of the experimental implementation. Neither the sequence of profiles nor the position of profiles (left versus right on a page) appear to have been consequential determinants of preferences.

### Results by Profile Order (Choice)

```{r split_by_profile_order1, fig.cap="Policy Support Results by Profile Order"}
plot(cj(conj, Q1 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ pair, estimate = "mm", level_order = "descending"),
     group = "BY", vline = 0.5, header_fmt = "%s") +
  theme(
    axis.text.y = element_text(face = bolding),
    axis.title = element_text(size = 14, face = "bold"),
    legend.text = element_text(size = 14)
  )
```

\clearpage

### Results by Profile Order (Legitimacy)

```{r split_by_profile_order2, fig.cap="Perceived Legitimacy Results by Profile Order"}
plot(cj(conj, Q4 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ pair, estimate = "mm", level_order = "descending"),
     group = "BY", vline = mean(conj$Q4, na.rm = TRUE), header_fmt = "%s") +
  theme(
    axis.text.y = element_text(face = bolding),
    axis.title = element_text(size = 14, face = "bold"),
    legend.text = element_text(size = 14)
  )
```

\clearpage

### Results by Left/Right Profile Position (Choice)

```{r split_by_profile_pos1, fig.cap="Policy Support Results by Profile Position"}
conj$Position <- factor(conj$AB, levels = c("A", "B"), labels = c("Left", "Right"))
plot(cj(conj, Q1 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Position, estimate = "mm", level_order = "descending"),
     group = "BY", vline = 0.5, header_fmt = "%s") +
  theme(
    axis.text.y = element_text(face = bolding),
    axis.title = element_text(size = 14, face = "bold"),
    legend.text = element_text(size = 14)
  )
conj$Position <- NULL
```

\clearpage

### Results by Left/Right Profile Position (Legitimacy)

```{r split_by_profile_pos2, fig.cap="Perceived Legitimacy Results by Profile Position"}
conj$Position <- factor(conj$AB, levels = c("A", "B"), labels = c("Left", "Right"))
plot(cj(conj, Q4 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Position, estimate = "mm", level_order = "descending"),
     group = "BY", vline = mean(conj$Q4, na.rm = TRUE), header_fmt = "%s") +
  theme(
    axis.text.y = element_text(face = bolding),
    axis.title = element_text(size = 14, face = "bold"),
    legend.text = element_text(size = 14)
  )
conj$Position <- NULL
```

\clearpage

### Results by Survey Response Duration (Choice)

```{r split_by_survey_time1, fig.cap="Policy Support Results by Survey Duration"}
conj$Duration <- as.numeric((conj$endtime - conj$starttime)/60)
conj$Duration <- factor(cut(conj$Duration, quantile(conj$Duration, c(0.0,0.5,1.0))))
levels(conj$Duration) <- c("Low", "High")
plot(cj(conj, Q1 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Duration, estimate = "mm", level_order = "descending"),
     group = "BY", vline = 0.5, header_fmt = "%s") +
  theme(
    axis.text.y = element_text(face = bolding),
    axis.title = element_text(size = 14, face = "bold"),
    legend.text = element_text(size = 14)
  )
```

\clearpage

### Results by Survey Response Duration (Legitimacy)

```{r split_by_survey_time2, fig.cap="Perceived Legitimacy Results by Survey Duration"}
plot(cj(conj, Q4 ~ immigration + laws + rights + budget + payment + trade + border + timeline, id = ~ ID, by = ~ Duration, estimate = "mm", level_order = "descending"),
     group = "BY", vline = mean(conj$Q4, na.rm = TRUE), header_fmt = "%s") +
  theme(
    axis.text.y = element_text(face = bolding),
    axis.title = element_text(size = 14, face = "bold"),
    legend.text = element_text(size = 14)
  )
conj$Duration <- NULL
```


\clearpage

# References

Auspurg K, and T Hinz. 2015. *Factorial Survey Experiments*. Los Angeles, CA: Sage. 

Bansak, K., Hainmueller, J., and Hangartner, D. 2016. "How economic, humanitarian, and religious concerns shape European attitudes toward asylum seekers." *Science*, 354(6309), 217-222. 

Gallego, A. and Marx, P., 2017. "Multi-dimensional preferences for labour market reforms: a conjoint experiment." *Journal of European Public Policy*, 24(7), pp.1027-1047.

Hainmueller, J., and Hopkins, D.J. 2015. "The hidden American immigration consensus: A conjoint analysis of attitudes toward immigrants." *American Journal of Political Science* 59: 529–48.

Hainmueller, J., Hopkins, D.J.  and Yamamoto, T.. 2014. "Causal inference in conjoint analysis: Understanding multidimensional choices via stated preferences experiments." *Political Analysis* 22(1): 1-30.

Jasso, G. 2006. "Factorial survey methods for studying beliefs and judgements." *Sociological Methods & Research* 34(3): 334-423. 

UK Department for Exiting the European Union. 2017. "The United Kingdom’s exit from and new partnership with the European Union White Paper." URL:  https://www.gov.uk/government/publications/the-united-kingdoms-exit-from-and-new-partnership-with-the-european-union-white-paper

Wallander L. 2009. "25 years of factorial surveys in sociology: A review." *Social Science Research* 38(3): 505-20. 



\clearpage

# Analytic Details

This report was built with rmarkdown, using the following software versions:

```{r sessioninfo}
sessionInfo()
```
